{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "tags": [
     "pdf-title"
    ]
   },
   "source": [
    "# Dropout\n",
    "Dropout [1] is a technique for regularizing neural networks by randomly setting some output activations to zero during the forward pass. In this exercise you will implement a dropout layer and modify your fully-connected network to optionally use dropout.\n",
    "\n",
    "[1] [Geoffrey E. Hinton et al, \"Improving neural networks by preventing co-adaptation of feature detectors\", arXiv 2012](https://arxiv.org/abs/1207.0580)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "tags": [
     "pdf-ignore"
    ]
   },
   "outputs": [],
   "source": [
    "# As usual, a bit of setup\n",
    "from __future__ import print_function\n",
    "import time\n",
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "from cs231n.classifiers.fc_net import *\n",
    "from cs231n.data_utils import get_CIFAR10_data\n",
    "from cs231n.gradient_check import eval_numerical_gradient, eval_numerical_gradient_array\n",
    "from cs231n.solver import Solver\n",
    "\n",
    "%matplotlib inline\n",
    "plt.rcParams['figure.figsize'] = (10.0, 8.0) # set default size of plots\n",
    "plt.rcParams['image.interpolation'] = 'nearest'\n",
    "plt.rcParams['image.cmap'] = 'gray'\n",
    "\n",
    "# for auto-reloading external modules\n",
    "# see http://stackoverflow.com/questions/1907993/autoreload-of-modules-in-ipython\n",
    "%load_ext autoreload\n",
    "%autoreload 2\n",
    "\n",
    "def rel_error(x, y):\n",
    "  \"\"\" returns relative error \"\"\"\n",
    "  return np.max(np.abs(x - y) / (np.maximum(1e-8, np.abs(x) + np.abs(y))))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "tags": [
     "pdf-ignore"
    ]
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "X_train:  (49000, 3, 32, 32)\n",
      "y_train:  (49000,)\n",
      "X_val:  (1000, 3, 32, 32)\n",
      "y_val:  (1000,)\n",
      "X_test:  (1000, 3, 32, 32)\n",
      "y_test:  (1000,)\n"
     ]
    }
   ],
   "source": [
    "# Load the (preprocessed) CIFAR10 data.\n",
    "\n",
    "data = get_CIFAR10_data()\n",
    "for k, v in data.items():\n",
    "  print('%s: ' % k, v.shape)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Dropout forward pass\n",
    "In the file `cs231n/layers.py`, implement the forward pass for dropout. Since dropout behaves differently during training and testing, make sure to implement the operation for both modes.\n",
    "\n",
    "Once you have done so, run the cell below to test your implementation."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Running tests with p =  0.25\n",
      "Mean of input:  10.000207878477502\n",
      "Mean of train-time output:  2.5035147792443206\n",
      "Mean of test-time output:  10.000207878477502\n",
      "Fraction of train-time output set to zero:  0.749784\n",
      "Fraction of test-time output set to zero:  0.0\n",
      "\n",
      "Running tests with p =  0.4\n",
      "Mean of input:  10.000207878477502\n",
      "Mean of train-time output:  3.991167063504464\n",
      "Mean of test-time output:  10.000207878477502\n",
      "Fraction of train-time output set to zero:  0.600796\n",
      "Fraction of test-time output set to zero:  0.0\n",
      "\n",
      "Running tests with p =  0.7\n",
      "Mean of input:  10.000207878477502\n",
      "Mean of train-time output:  6.9914683385116\n",
      "Mean of test-time output:  10.000207878477502\n",
      "Fraction of train-time output set to zero:  0.30074\n",
      "Fraction of test-time output set to zero:  0.0\n",
      "\n"
     ]
    }
   ],
   "source": [
    "np.random.seed(231)\n",
    "x = np.random.randn(500, 500) + 10\n",
    "\n",
    "for p in [0.25, 0.4, 0.7]:\n",
    "  out, _ = dropout_forward(x, {'mode': 'train', 'p': p})\n",
    "  out_test, _ = dropout_forward(x, {'mode': 'test', 'p': p})\n",
    "\n",
    "  print('Running tests with p = ', p)\n",
    "  print('Mean of input: ', x.mean())\n",
    "  print('Mean of train-time output: ', out.mean())\n",
    "  print('Mean of test-time output: ', out_test.mean())\n",
    "  print('Fraction of train-time output set to zero: ', (out == 0).mean())\n",
    "  print('Fraction of test-time output set to zero: ', (out_test == 0).mean())\n",
    "  print()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Dropout backward pass\n",
    "In the file `cs231n/layers.py`, implement the backward pass for dropout. After doing so, run the following cell to numerically gradient-check your implementation."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "dx relative error:  1.892896954038074e-11\n"
     ]
    }
   ],
   "source": [
    "np.random.seed(231)\n",
    "x = np.random.randn(10, 10) + 10\n",
    "dout = np.random.randn(*x.shape)\n",
    "\n",
    "dropout_param = {'mode': 'train', 'p': 0.2, 'seed': 123}\n",
    "out, cache = dropout_forward(x, dropout_param)\n",
    "dx = dropout_backward(dout, cache)\n",
    "dx_num = eval_numerical_gradient_array(lambda xx: dropout_forward(xx, dropout_param)[0], x, dout)\n",
    "\n",
    "# Error should be around e-10 or less\n",
    "print('dx relative error: ', rel_error(dx, dx_num))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "tags": [
     "pdf-inline"
    ]
   },
   "source": [
    "## Inline Question 1:\n",
    "What happens if we do not divide the values being passed through inverse dropout by `p` in the dropout layer? Why does that happen?\n",
    "\n",
    "## Answer:\n",
    "dropout 完了要恢复的啊\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Fully-connected nets with Dropout\n",
    "In the file `cs231n/classifiers/fc_net.py`, modify your implementation to use dropout. Specifically, if the constructor of the network receives a value that is not 1 for the `dropout` parameter, then the net should add a dropout layer immediately after every ReLU nonlinearity. After doing so, run the following to numerically gradient-check your implementation."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Running check with dropout =  1\n",
      "Initial loss:  2.3004790897684924\n",
      "W1 relative error: 1.48e-07\n",
      "W2 relative error: 2.21e-05\n",
      "W3 relative error: 3.53e-07\n",
      "b1 relative error: 5.38e-09\n",
      "b2 relative error: 2.09e-09\n",
      "b3 relative error: 5.80e-11\n",
      "\n",
      "Running check with dropout =  0.75\n",
      "Initial loss:  2.302454105929313\n",
      "W1 relative error: 2.56e-07\n",
      "W2 relative error: 2.10e-06\n",
      "W3 relative error: 5.57e-08\n",
      "b1 relative error: 6.43e-09\n",
      "b2 relative error: 2.04e-09\n",
      "b3 relative error: 1.10e-10\n",
      "\n",
      "Running check with dropout =  0.5\n",
      "Initial loss:  2.3030067920900876\n",
      "W1 relative error: 1.22e-06\n",
      "W2 relative error: 9.41e-08\n",
      "W3 relative error: 2.67e-07\n",
      "b1 relative error: 3.53e-08\n",
      "b2 relative error: 3.37e-09\n",
      "b3 relative error: 1.01e-10\n",
      "\n"
     ]
    }
   ],
   "source": [
    "np.random.seed(231)\n",
    "N, D, H1, H2, C = 2, 15, 20, 30, 10\n",
    "X = np.random.randn(N, D)\n",
    "y = np.random.randint(C, size=(N,))\n",
    "\n",
    "for dropout in [1, 0.75, 0.5]:\n",
    "  print('Running check with dropout = ', dropout)\n",
    "  model = FullyConnectedNet([H1, H2], input_dim=D, num_classes=C,\n",
    "                            weight_scale=5e-2, dtype=np.float64,\n",
    "                            dropout=dropout, seed=123)\n",
    "\n",
    "  loss, grads = model.loss(X, y)\n",
    "  print('Initial loss: ', loss)\n",
    "  \n",
    "  # Relative errors should be around e-6 or less; Note that it's fine\n",
    "  # if for dropout=1 you have W2 error be on the order of e-5.\n",
    "  for name in sorted(grads):\n",
    "    f = lambda _: model.loss(X, y)[0]\n",
    "    grad_num = eval_numerical_gradient(f, model.params[name], verbose=False, h=1e-5)\n",
    "    print('%s relative error: %.2e' % (name, rel_error(grad_num, grads[name])))\n",
    "  print()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Regularization experiment\n",
    "As an experiment, we will train a pair of two-layer networks on 500 training examples: one will use no dropout, and one will use a keep probability of 0.25. We will then visualize the training and validation accuracies of the two networks over time."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "scrolled": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "1\n",
      "(Iteration 1 / 125) loss: 7.856643\n",
      "(Epoch 0 / 25) train acc: 0.236000; val_acc: 0.190000\n",
      "(Epoch 1 / 25) train acc: 0.250000; val_acc: 0.178000\n",
      "(Epoch 2 / 25) train acc: 0.360000; val_acc: 0.217000\n",
      "(Epoch 3 / 25) train acc: 0.508000; val_acc: 0.242000\n",
      "(Epoch 4 / 25) train acc: 0.530000; val_acc: 0.244000\n",
      "(Epoch 5 / 25) train acc: 0.578000; val_acc: 0.283000\n",
      "(Epoch 6 / 25) train acc: 0.604000; val_acc: 0.253000\n",
      "(Epoch 7 / 25) train acc: 0.682000; val_acc: 0.263000\n",
      "(Epoch 8 / 25) train acc: 0.716000; val_acc: 0.282000\n",
      "(Epoch 9 / 25) train acc: 0.794000; val_acc: 0.304000\n",
      "(Epoch 10 / 25) train acc: 0.840000; val_acc: 0.271000\n",
      "(Epoch 11 / 25) train acc: 0.852000; val_acc: 0.283000\n",
      "(Epoch 12 / 25) train acc: 0.910000; val_acc: 0.286000\n",
      "(Epoch 13 / 25) train acc: 0.914000; val_acc: 0.307000\n",
      "(Epoch 14 / 25) train acc: 0.938000; val_acc: 0.310000\n",
      "(Epoch 15 / 25) train acc: 0.956000; val_acc: 0.298000\n",
      "(Epoch 16 / 25) train acc: 0.906000; val_acc: 0.285000\n",
      "(Epoch 17 / 25) train acc: 0.974000; val_acc: 0.319000\n",
      "(Epoch 18 / 25) train acc: 0.990000; val_acc: 0.290000\n",
      "(Epoch 19 / 25) train acc: 0.974000; val_acc: 0.281000\n",
      "(Epoch 20 / 25) train acc: 0.978000; val_acc: 0.295000\n",
      "(Iteration 101 / 125) loss: 0.236060\n",
      "(Epoch 21 / 25) train acc: 0.988000; val_acc: 0.296000\n",
      "(Epoch 22 / 25) train acc: 0.990000; val_acc: 0.289000\n",
      "(Epoch 23 / 25) train acc: 0.988000; val_acc: 0.287000\n",
      "(Epoch 24 / 25) train acc: 0.996000; val_acc: 0.293000\n",
      "(Epoch 25 / 25) train acc: 0.998000; val_acc: 0.296000\n",
      "\n",
      "0.25\n",
      "(Iteration 1 / 125) loss: 4.897379\n",
      "(Epoch 0 / 25) train acc: 0.222000; val_acc: 0.189000\n",
      "(Epoch 1 / 25) train acc: 0.322000; val_acc: 0.210000\n",
      "(Epoch 2 / 25) train acc: 0.374000; val_acc: 0.226000\n",
      "(Epoch 3 / 25) train acc: 0.436000; val_acc: 0.308000\n",
      "(Epoch 4 / 25) train acc: 0.514000; val_acc: 0.308000\n",
      "(Epoch 5 / 25) train acc: 0.490000; val_acc: 0.291000\n",
      "(Epoch 6 / 25) train acc: 0.540000; val_acc: 0.297000\n",
      "(Epoch 7 / 25) train acc: 0.604000; val_acc: 0.293000\n",
      "(Epoch 8 / 25) train acc: 0.576000; val_acc: 0.271000\n",
      "(Epoch 9 / 25) train acc: 0.596000; val_acc: 0.303000\n",
      "(Epoch 10 / 25) train acc: 0.642000; val_acc: 0.300000\n",
      "(Epoch 11 / 25) train acc: 0.670000; val_acc: 0.311000\n",
      "(Epoch 12 / 25) train acc: 0.610000; val_acc: 0.287000\n",
      "(Epoch 13 / 25) train acc: 0.718000; val_acc: 0.325000\n",
      "(Epoch 14 / 25) train acc: 0.706000; val_acc: 0.311000\n",
      "(Epoch 15 / 25) train acc: 0.744000; val_acc: 0.300000\n",
      "(Epoch 16 / 25) train acc: 0.734000; val_acc: 0.298000\n",
      "(Epoch 17 / 25) train acc: 0.780000; val_acc: 0.323000\n",
      "(Epoch 18 / 25) train acc: 0.764000; val_acc: 0.324000\n",
      "(Epoch 19 / 25) train acc: 0.798000; val_acc: 0.314000\n",
      "(Epoch 20 / 25) train acc: 0.824000; val_acc: 0.302000\n",
      "(Iteration 101 / 125) loss: 13.067745\n",
      "(Epoch 21 / 25) train acc: 0.842000; val_acc: 0.323000\n",
      "(Epoch 22 / 25) train acc: 0.852000; val_acc: 0.308000\n",
      "(Epoch 23 / 25) train acc: 0.854000; val_acc: 0.298000\n",
      "(Epoch 24 / 25) train acc: 0.892000; val_acc: 0.318000\n",
      "(Epoch 25 / 25) train acc: 0.894000; val_acc: 0.293000\n",
      "\n"
     ]
    }
   ],
   "source": [
    "# Train two identical nets, one with dropout and one without\n",
    "np.random.seed(231)\n",
    "num_train = 500\n",
    "small_data = {\n",
    "  'X_train': data['X_train'][:num_train],\n",
    "  'y_train': data['y_train'][:num_train],\n",
    "  'X_val': data['X_val'],\n",
    "  'y_val': data['y_val'],\n",
    "}\n",
    "\n",
    "solvers = {}\n",
    "dropout_choices = [1, 0.25]\n",
    "for dropout in dropout_choices:\n",
    "  model = FullyConnectedNet([500], dropout=dropout)\n",
    "  print(dropout)\n",
    "\n",
    "  solver = Solver(model, small_data,\n",
    "                  num_epochs=25, batch_size=100,\n",
    "                  update_rule='adam',\n",
    "                  optim_config={\n",
    "                    'learning_rate': 5e-4,\n",
    "                  },\n",
    "                  verbose=True, print_every=100)\n",
    "  solver.train()\n",
    "  solvers[dropout] = solver\n",
    "  print()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAA4IAAAJNCAYAAABkwMu0AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvnQurowAAIABJREFUeJzs3XuYXXV58P3vbRhkPGU4RISZAEHSICaB4BCtCCIUAyIhgI2gYrFV1IpY2ycCfWuemLdWBCq+FJ76pB4AD0BKYwwVjAoeilbMxOBEgpGAYGZAJUDioQOEcL9/7D1hZpgcJnv2XrP3/n6ua669173WXuvemc0w9/x+6/5FZiJJkiRJah7PKzoBSZIkSVJtWQhKkiRJUpOxEJQkSZKkJmMhKEmSJElNxkJQkiRJkpqMhaAkSZIkNRkLQUlSU4iIcRHxh4g4oOhcJEkqWriOoCRpLIqIPwzYfAHwJLClvP3ezPxy7bOSJKkxWAhKksa8iHgAeHdmfns7x+yWmU/XLqvaaeT3JkkqhlNDJUl1KSL+MSJujIjrI+L3wDsi4k8j4kcRsTEiHo6IKyOipXz8bhGREXFQeftL5f23RsTvI+K/I2LSNq71vIi4KSJ+XT73dyPiFQP2vyAiroiIX0XEpoj4fkQ8v7zv2HJOmyJifUScU47fERHnDjjHuyPiu0Ny/euIWAf8vBy/KiJ6IuJ3EbEiIl474PW7RcRHI+K+8v6uiNg/Iv5vRHxyyPu5JSI+WPl3QZJUrywEJUn17HTgK8B44EbgaeBDwD7A0cBJwHu38/q3AR8F9gJ+Bfy/2zn2P4HJwMuAnwFfHLDvCmA68Oryuf4eeKZcWN4CfArYG5gBrB7B+5sNHAVMK2/fWb7OXsBNwL/3F5zAPOAtlN5zG/Bu4AngWuDsiAiAiNgXOA64fgR5SJIajIWgJKme3ZGZN2fmM5nZl5krMvPOzHw6M+8HFgGv387rb8rMrszcDHwZOGK4g8rnvyYzf5+ZTwALgFdFxAsjYhxwLnBBZj6cmVsy847yOd8B3JqZi8s5bcjMu0bw/v4pMx/PzL5yHl/MzMfK00QvBV4CHFI+9t3A32fmveV87yof+0NKBeFx5ePOBr6dmRtGkIckqcFYCEqS6tn6gRsRcWhEfL08hfN3wEJKo4Pb8usBz/8HeNFwB5U7jl4aEfeXz7uuvGsfYF9gd+C+YV46cRvxnTX0/X0kIn4eEZuAx4EX8uz72961rqNUlFJ+/OI2jpMkNQkLQUlSPRva8ez/Upq2eUhmvgSYD8QoXOedwJuA4ylNQ+0fhQvgN8BTwMuHed36bcQB/kipG2q/lw1zzNb3FxFvAP4WOJPS1M89gT/w7Pvb3rW+CJweETPKx9y8jeMkSU3CQlCS1EheDGwC/lhu5rK9+wNHet4ngUcpFW8f79+RmVuAa4BPR8TLyqOHR5eb1HwJOCkiziw3c9knIg4vv/Qu4MyIaI2IPwH+cidyeBrYALRQmp76wgH7Pwv8Y0S8PEqOiIi9yjk+CPyU0v2C/16e3ipJamIWgpKkRvJ3wF8Av6c0OnjjKJ33C8BD5a+7gR8O2f9h4B5gJfAY8E+Ulmj6JXAqcGE5/hOebfxyOaURv98Cn6dUNG7PLcC3gXuBB4DfAQ8P2H8ZsBS4rbxvEbDHgP3Xlq/ttFBJkusISpLUDCLieOBzwMHp//wlqek5IihJUoOLiN0pLavxbxaBkiSwEJQkqaFFxDRKHUb3Aq4sOB1J0hjh1FBJkiRJajKOCEqSJElSk7EQlCRJkqQms1vRCYyWffbZJw866KCi05AkSZKkQqxcuXJDZk7YmWMbphA86KCD6OrqKjoNSZIkSSpERDy4s8c6NVSSJEmSmoyFoCRJkiQ1GQtBSZIkSWoyFoKSJEmS1GQsBCVJkiSpyRRSCEbE5yPitxHxs23sj4i4MiLWRUR3RBxZ6xwlSZIkqVEVtXzENcBVwHXb2H8yMLn89WrgX8uPkiRJklSIpat6uWz5Wh7a2Mf+ba3MmzWFOTPai05rlxRSCGbm9yPioO0cchpwXWYm8KOIaIuI/TLz4ZokKEmSJGnMKqIgW7qql4uXrKZv8xYAejf2cfGS1QB1WQyO1QXl24H1A7Z7yjELQUmSpJ3USKMX2rZm+z4XVZBdtnzt1mv269u8hcuWr63Lf++xWgjulIg4DzgP4IADDig4G0mSpLGj0UYv6kEzjlIV8Z6LKsge2tg3ovhYN1a7hvYCEwdsd5Rjg2TmoszszMzOCRMm1Cw5SZKksW57vyxr9PUXZL0b+0ieLciWrnrOr7Cjqsjvc1HvuaiCbP+21hHFx7qxWgguA95Z7h76GmCT9wdKkiTtvEYbvRjriirIivw+F/WeiyrI5s2aQmvLuEGx1pZxzJs1parXrZZCpoZGxPXAccA+EdED/G+gBSAzPwPcArwJWAf8D/CuIvKUJEmqV/u3tdI7TDFQr6MXO6uo++WKHKUq6vtc1HueN2vKoOmwUJuCrP9z1Cj3YxbVNfTsHexP4AM1SkeSJKmqiihOivpluV+z3S9XVEFW5Pe5qPdcZEE2Z0Z73RZ+Q9V1sxhJkqSxrqjipMhflpuxq2MzjlIVWYQ2UkFWFAtBSZKkKiqyOCnql+Vm7OrYjKNUjTZVstlYCEqSJFVRMzZtacb75aA5R6ma8T03irHaNVSSJKkhNFrL+Z1hV0dp7LMQlCRJNbV0VS9HX3I7ky76OkdfcnvV1xwrWjMWJ0W95zkz2vnEGdNob2slgPa2Vj5xxjRHrKRhODVUkiTVTJFdHYvSjPdRNeP9clK9idJKDfWvs7Mzu7q6ik5DkiRtx9GX3D7sPVztba384KLjC8hIkhpHRKzMzM6dOdapoZIkqWaasXGKJI1FFoKSJKlmmrFxiiSNRRaCkiQ1qSKatjRj4xRJGotsFiNJUhMqqmlLMzZOkaSxyEJQkqQmdNnytVuLwH59m7dw2fK1VS/K7OooScVzaqgkSU3Ipi2S1NwsBCVJakI2bZGk5mYhKElSE7JpiyQ1N+8RlCSpCdm0RZKam4WgJEkFWrqqt7BizKYtktS8LAQlSaKYgqyoJRwkSbIQlCQ1vaIKsiKXcGhGRY6+StJYU0izmIg4KSLWRsS6iLhomP0HRsRtEdEdEd+NiI4i8pQkNYftFWTV5BIOtdNf7Pdu7CN5tthfuqq36NQkqRA1LwQjYhxwNXAycBhwdkQcNuSwy4HrMnM6sBD4RG2zlCQ1k6IKMpdwqJ2iin1JGquKGBGcCazLzPsz8yngBuC0IcccBtxefv6dYfZLkjRqiirIXMKhdhx9lTQquhfDFVNhQVvpsXtx0RntsiIKwXZg/YDtnnJsoJ8CZ5Sfnw68OCL2rkFukqQmVFRBNmdGO584Yxrtba0E0N7WyifOmOZ9a1Xg6KvUYIooyLoXw80XwKb1QJYeb76gbovBsdos5n8BV0XEucD3gV5gy9CDIuI84DyAAw44oJb5SZIaSJFr6rmEQ23MmzVlUEMgcPRVqlv9Bdnm8oh+f0EGMH1u9a5728Jnr9lvc18pXs3rVkkRhWAvMHHAdkc5tlVmPkR5RDAiXgScmZkbh54oMxcBiwA6OzuzWglLkhqfBVljK7LYlzTKiirINvWMLD7GFVEIrgAmR8QkSgXgWcDbBh4QEfsAj2XmM8DFwOdrnqUkSWooFvtSgyiqIBvfUZ4WOky8DtX8HsHMfBo4H1gO3AMszsy7I2JhRMwuH3YcsDYifgHsC3y81nlKkiRJGoO2VXhVuyA7YT60DLmvuKW1FK9DhdwjmJm3ALcMic0f8Pwm4KZa5yVJkiRpjDth/uB7BKE2BVn/tNPbFpZGH8d3lK5Zh/cHwthtFiNJkiRJz1VkQTZ9bt0WfkNZCEqSJEmqLw1UkBWliHUEJUmSJEkFshCUJEmSpCZjIShJkiRJTcZ7BCVJY8bSVb0u+C1JI9W9uLhOlkVeWxWxEJQkjQlLV/Vy8ZLV9G3eAkDvxj4uXrIawGJQkrale/HgpRQ2rS9tQ/ULsiKvrYo5NVSSNCZctnzt1iKwX9/mLVy2fG1BGUlSHbht4eD19KC0fdvCxr62KuaIoCTpOYqYovnQxr4RxSVpzClimuSmnpHFG+XaqpgjgpKkQfqnaPZu7CN5dorm0lW9Vb3u/m2tI4pL0pjSP01y03ogn50m2b24utcd3zGyeKNcWxWzEJQkDVLUFM15s6bQ2jJuUKy1ZRzzZk2p6nUlaVQUNU3yhPnQMuQPZi2tpXi1FXltVcypoZKkQYqaotk/9dSuoZLqUlHTJPunnhbRubPIa6tiFoKSpEH2b2uld5iirxZTNOfMaLfwk1SfxneUp4UOE6+26XOLK76KvLYq4tRQSdIgTtGUVNe6F8MVU2FBW+mx2vfo9XOapOqMI4KSpEGcoimpbhW5rp3TJFVnIjOLzmFUdHZ2ZldXV9FpSJIkqShXTN3G9MyJ8OGf1T4fqcYiYmVmdu7MsU4NlSRJUmNwXTtpp1kISpIkqTG4rp200ywEJUmS1Bhs2CLttEIKwYg4KSLWRsS6iLhomP0HRMR3ImJVRHRHxJuKyFOSJEl1ZPpcOPXK0j2BROnx1Ctt2CINo+ZdQyNiHHA1cCLQA6yIiGWZuWbAYf8ALM7Mf42Iw4BbgINqnaskSZJ2Uffi4hY5t/CTdqiI5SNmAusy836AiLgBOA0YWAgm8JLy8/HAQzXNUJLGgKWrel3CQVJ9KnIZB0k7pYipoe3AwL6+PeXYQAuAd0RED6XRwA/WJjVJGhuWrurl4iWr6d3YRwK9G/u4eMlqlq7qLTo1Sdqx2xY+WwT229xXiksaE8Zqs5izgWsyswN4E/DFiHhOrhFxXkR0RUTXI488UvMkJalaLlu+lr7NWwbF+jZv4bLlawvKSJJGwGUcpDGviEKwF5g4YLujHBvor4DFAJn538AewD5DT5SZizKzMzM7J0yYUKV0Jan2HtrYN6K4JI0pLuMgjXlFFIIrgMkRMSkidgfOApYNOeZXwAkAEfEKSoWgQ36Smsb+ba0jiksa47oXwxVTYUFb6bF7cdEZVZfLOEhjXs0Lwcx8GjgfWA7cQ6k76N0RsTAiZpcP+zvgPRHxU+B64NzMzFrnKklFmTdrCq0t4wbFWlvGMW/WlIIykrTL+hunbFoP5LONUxq5GHQZB2nMi0aprzo7O7Orq6voNCRp1Ng1VGoQV0wtF4FDjJ8IH/5Z7fOR1LAiYmVmdu7MsUUsHyFJ2glzZrRb+EmNwMYpksagsdo1VJIkqTHYOEXSGGQhKEmSVE1FNk5ptiY1knaaU0MlSZKqqb9Bym0LS9NBx3eUisBqN07pb1LTv7B7f5OagTlJaloWgpIkSdU2fW7ti6/bFj5bBPbb3FeKWwhKTc+poZIkSY3IJjWStsNCUJIkqRHZpEbSdlgISpKk2rKBSW0U2aRG0pjnPYKSJKl2bGBSO0U1qZFUFywEJUlS7djApLaKaFIjqS44NVSSJNVOkQ1MnJIqSVtZCEqSpNopqoFJ/5TUTeuBfHZKqsWgpCZlIShJkmqnqAYm25uSKklNyEJQkiTVzvS5cOqVMH4iEKXHU6+s/n1srqknSYPYLEaSJNVWEQ1MxneUp4UOE5ekJuSIoCTtwNJVvRx9ye1MuujrHH3J7Sxd1Vt0SmokNjCpDdfUk6RBHBGUpO1YuqqXi5espm/zFgB6N/Zx8ZLVAMyZ0V5kamoErqlXO66pJ0mDRGYWncOo6OzszK6urqLTkNRgjr7kdno39j0n3t7Wyg8uOr6AjNRQrpi6jemKE+HDP6t9PpKkuhYRKzOzc2eOdWqoJG3HQ8MUgduLSyNSdAMTp6VKUtMqpBCMiJMiYm1ErIuIi4bZf0VE3FX++kVEbCwiT0nav611RHFpRIpaUw9cV0+SmlzNC8GIGAdcDZwMHAacHRGHDTwmMz+cmUdk5hHAvwBLap2nJAHMmzWF1pZxg2KtLeOYN2tKQRmpoRTZwMR19SSpqRUxIjgTWJeZ92fmU8ANwGnbOf5s4PqaZCZJQ8yZ0c4nzphGe1srQenewE+cMc1GMRodRa2pB8VPS5UkFaqIrqHtwMA743uAVw93YEQcCEwCbq9BXpI0rDkz2i38VD1FrKkHrqsnSU1urDeLOQu4KTO3DLczIs6LiK6I6HrkkUdqnJokSXXMdfUkqakVUQj2AhMHbHeUY8M5i+1MC83MRZnZmZmdEyZMGMUUJUlqcEVOS5UkFa6IqaErgMkRMYlSAXgW8LahB0XEocCewH/XNj1JY9XSVb1ctnwtD23sY/+2VubNmuKUTakSRU1LlSQVruaFYGY+HRHnA8uBccDnM/PuiFgIdGXmsvKhZwE3ZKOseC+pIktX9XLxktX0bS7NFO/d2MfFS1YDWAxKkiSNUFRSZ0XEB4EvZebjo5fSruns7Myurq6i05BUJUdfcju9wyzi3t7Wyg8uOr6AjCRJksaWiFiZmZ07c2yl9wjuC6yIiMXlReKjwvNJ0rAeGqYI3F5cdap7MVwxFRa0lR5d3FySpKqoqBDMzH8AJgOfA84F7o2If4qIl49CbpK01f5trSOKqw51L4abLygvaZClx5svsBiUJKkKKu4aWr6H79flr6cpNXi5KSIurfTcktRv3qwptLaMGxRrbRnHvFlTCspIo+62hbB5yAjv5r5SXJIkjaqKmsVExIeAdwIbgM8C8zJzc0Q8D7gX+EjlKUrSsw1h7BrawDb1jCwuSZJ2WaVdQ/cCzsjMBwcGM/OZiHhzheeWpEHmzGi38Gtk4zvK00KHiUuSpFFV6dTQW4HH+jci4iUR8WqAzLynwnNLkprJCfOhZcg9ny2tpbgkSRpVlRaC/wr8YcD2H8oxSZJGZvpcOPVKGD8RiNLjqVe64LkkSVVQ6dTQGLjge3lKaM0XqZckNYjpcy38JEmqgUpHBO+PiAsioqX89SHg/tFITJJUINfzkySpoVVaCL4PeC3QC/QArwbOqzQpSVKBXM9PkqSGV9E0zsz8LXDWKOUiSRoLtreen9M2JUlqCJWuI7gH8FfAK4E9+uOZ+ZcV5iVJKorr+UmS1PAqnRr6ReBlwCzge0AH8PtKk5IkFWhb6/a5np8kSQ2j0kLwkMz8KPDHzLwWOIXSfYKSpHrlen6SJDW8SgvBzeXHjRExFRgPvLTCc0qSiuR6fpIkNbxK1/xbFBF7Av8ALANeBHy04qwkScVyPT9JkhraLheCEfE84HeZ+TjwfeDgUctKkiRJklQ1uzw1NDOfAT4yirlIklSc7sVwxVRY0FZ6dN1ESVIDq3Rq6Lcj4n8BNwJ/7A9m5mMVnleSpNrpXgw3X/Ds+omb1pe2wSmykqSGVGmzmLcCH6A0NXRl+atrRy+KiJMiYm1ErIuIi7ZxzNyIWBMRd0fEVyrMU9IoWbqql6MvuZ1JF32doy+5naWremt3cUdsVC23LXy2COy3ua8UlySpAVU0IpiZk0b6mogYB1wNnAj0ACsiYllmrhlwzGTgYuDozHw8IuxEKo0BS1f1cvGS1fRt3gJA78Y+Ll6yGoA5M9qre3FHbFRNm3pGFpckqc5VVAhGxDuHi2fmddt52UxgXWbeXz7HDcBpwJoBx7wHuLrciIbM/G0leUoaHZctX7u1COzXt3kLly1fW/1CcHsjNo1aCHYvLr2/TT2lxdxPmN+477Vo4ztKf1wYLi5JUgOqdGroUQO+jgEWALN38Jp2YOD/bXvKsYH+BPiTiPhBRPwoIk6qME9Jo+ChjX0jio+qZhux6R8B3bQeyGdHQJ0OWx0nzIeW1sGxltZSXJKkBlTp1NAPDtyOiDbghooyKtkNmAwcB3QA34+IaZm5ccj1zgPOAzjggANG4bKStmf/tlZ6hyn69m9rHeboUdZsIzbNOAJapP5/U0dgJUlNotIRwaH+COzovsFeYOKA7Y5ybKAeYFlmbs7MXwK/oFQYDpKZizKzMzM7J0yYUEHaknbGvFlTaG0ZNyjW2jKOebOmVP/izTZi02wjoGPB9Lnw4Z/Bgo2lR4tASVIDq/QewZuBLG8+DzgM2NG8pRXA5IiYRKkAPAt425BjlgJnA1+IiH0oTRW9v5JcJVWu/z7Ay5av5aGNfezf1sq8WVOqf38gNN+ITbONgEqSpJqqdB3Bywc8fxp4MDO3++fqzHw6Is4HlgPjgM9n5t0RsRDoysxl5X1vjIg1wBZgXmY+WmGuUkNZuqq3kIJszoz22hR+w5k+t3ELv6FOmD+4Syo09gioJEmqqcjMHR+1rReXRvUezswnytutwL6Z+cDopLfzOjs7s6trh0sYSg1h6DIOUJqi+YkzphVXpDWyorp32jVUkiSNQESszMzOnTm20hHBfwdeO2B7Szl2VIXnlbQdhS7j0GyKXL+wmUZAJUlSTVXaLGa3zHyqf6P8fPcKzylpBwpdxqHZbK97pyRJUp2qtBB8JCK2rhsYEacBGyo8p6Qd2NZyDTVZxqHZ2L1TkiQ1oEoLwfcBfx8Rv4qIXwEXAu+tPC1J21PoMg7NZltdOu3eKUmS6lilC8rfB7wmIl5U3v7DqGQlabsKXcah2di9U5IkNaBK1xH8J+DSzNxY3t4T+LvM/IfRSE7SthW6jEMzabb1CyVJUlOodPmIVZk5Y0jsJ5l5ZMWZjZDLR0iSJElqZiNZPqLSewTHRcTzB1y4FXj+do6XJEmSJBWs0nUEvwzcFhFfAAI4F7i20qQkSZIkSdVTabOYT0bET4E/AxJYDhw4GolJGqO6F3u/nCRJUp2rdEQQ4DeUisA/B34J/MconFPSWNS9eHAHzU3rS9tgMShJklRHdqkQjIg/Ac4uf20AbqTUeOYNo5ibpLHmtoWDl1GA0vZtCy0EJUmS6siujgj+HPgv4M2ZuQ4gIj48allJdWLpqt7mWstvU8/I4pIkSRqTdrVr6BnAw8B3IuLfIuIESs1ipKaxdFUvFy9ZTe/GPhLo3djHxUtWs3RVb9GpVc/4jpHFJUmSNCbtUiGYmUsz8yzgUOA7wN8AL42If42IN45mgtJYddnytfRt3jIo1rd5C5ctX1tQRjVwwnxoaR0ca2ktxSVJklQ3KlpHMDP/mJlfycxTgQ5gFXDhqGQmjXEPbewbUbwhTJ8Lp14J4ycCUXo89UrvD5QkSaozo9E1FIDMfBxYVP6SGt7+ba286nff4iO7LWb/2MBDuQ+XPj2XlS85sejUqmv6XAs/SZKkOjdqhaDUbD592L1MXflZWuMpADpiA59s+Sw/O+wg4PhCc5MkSZK2x0JQDaGI7p1H3fcvUC4C+7XGU6U4763qtSVJkqRKVHSP4K6KiJMiYm1ErIuIi4bZf25EPBIRd5W/3l1EnqoPhXXvdCkFSZIk1amaF4IRMQ64GjgZOAw4OyIOG+bQGzPziPLXZ2uapOpKYd07XUpBkiRJdaqIEcGZwLrMvD8znwJuAE4rIA81iMK6d7qUgiRJkupUEYVgO7B+wHZPOTbUmRHRHRE3RcTE2qSmerR/W+uI4qPGpRQkSZJUp8Zqs5ibgesz88mIeC9wLcO0YYyI84DzAA444IDaZqgxY96sKVy8ZPWg6aGtLeOYN2tK9S9e5FIK3YvhtoWlexLHd5RGIi1CJUmStBOKGBHsBQaO8HWUY1tl5qOZ+WR587PAq4Y7UWYuyszOzOycMGFCVZLV2DdnRjufOGMa7W2tBNDe1sonzphW9a6hhepeDDdfAJvWA1l6vPmCUlySJEnagSJGBFcAkyNiEqUC8CzgbQMPiIj9MvPh8uZs4J7apqh6M2dGe2MXfkPdthA2D7kHcnNfKe6ooCRJknag5oVgZj4dEecDy4FxwOcz8+6IWAh0ZeYy4IKImA08DTwGnFvrPKUxzaUrJEmSVIFC7hHMzFuAW4bE5g94fjFwca3zkurG+I7ytNBh4pIkSdIOFLKgvKQKuXSFJEmSKmAhKNUjl66QJElSBcbq8hGSdqTIpSskSZJU1xwRlCRJkqQmYyEoSZIkSU3GQlCSJEmSmoyFoCRJkiQ1GQtBSZIkSWoyFoKSJEmS1GQsBCVJkiSpyVgIqjF0L4YrpsKCttJj9+KiM5IkSZLGLBeUV/3rXgw3XwCb+0rbm9aXtsEF1yVJkqRhOCKo+nfbwmeLwH6b+0pxSZIkSc9hIaj6t6lnZHFJkiSpyVkIqv6N7xhZXJIkSWpyFoKqfyfMh5bWwbGW1lJckiRJ0nNYCKr+TZ8Lp14J4ycCUXo89UobxUiSJEnbYNdQNYbpcy38JEmSpJ3kiKAkSZIkNZlCCsGIOCki1kbEuoi4aDvHnRkRGRGdtcxPkiRJkhpZzQvBiBgHXA2cDBwGnB0Rhw1z3IuBDwF31jZDSZIkSWpsRYwIzgTWZeb9mfkUcANw2jDH/b/AJ4EnapmcJEmSJDW6IgrBdmD9gO2ecmyriDgSmJiZX69lYpIkSZLUDMZcs5iIeB7wKeDvduLY8yKiKyK6HnnkkeonJ0mSJEkNoIhCsBeYOGC7oxzr92JgKvDdiHgAeA2wbLiGMZm5KDM7M7NzwoQJVUxZkiRJkhpHEYXgCmByREyKiN2Bs4Bl/Tszc1Nm7pOZB2XmQcCPgNmZ2VVArpIkSZLUcGpeCGbm08D5wHLgHmBxZt4dEQsjYnat85EkSZKkZrNbERfNzFuAW4bE5m/j2ONqkZNGQfdiuG0hbOqB8R1wwnyYPrforCRJkiQNUUghqAbUvRhuvgA295W2N60vbYPFoCRJkjTGjLmuoapTty18tgjst7mvFJckSZI0plgIanRs6hlZXJIkSVJhLAQ1OsZ3jCwuSZIkqTAWghoVK17+Qfpy90GxvtydFS//YEEZSZIkSdoWC0GNir9ZM5kLN7+bnmf24ZkMep7Zhws3v5u/WTO56NQkSZIkDWHXUI2Khzb20cvrWPbU6wbFY2PfNl4hSZIkqSiOCGpU7N/WOqK4JEmSpOJYCGpUzJs1hdaWcYNirS3jmDdrSkEZSZIkSdoWp4ZqVMyZ0Q7AZcvX8tDGPvZva2XerClb45IkSZLGDgtBjZo5M9ot/CRJkqQ64NRQSZIkSWoyFoKSJEmS1GQsBCVJkiSpyVgINqLuxXDFVFjQVnrsXlx0RpKRBe9DAAAgAElEQVQkSZLGEJvFNJruxXDzBbC5vJD7pvWlbYDpc4vLS5IkSdKY4Yhgo7lt4bNFYL/NfaW4JEmSJGEh2Hg29YwsLkmSJKnpWAg2mvEdI4tLkiRJajqFFIIRcVJErI2IdRFx0TD73xcRqyPiroi4IyIOKyLPunTCfGhpHRxraS3FJUmSJIkCCsGIGAdcDZwMHAacPUyh95XMnJaZRwCXAp+qcZr1a/pcOPVKGD8RiNLjqVfaKEaSJEnSVkV0DZ0JrMvM+wEi4gbgNGBN/wGZ+bsBx78QyJpmWO+mz7XwkyRJkrRNRRSC7cD6Ads9wKuHHhQRHwD+FtgdOL42qUmSJElS4xuzzWIy8+rMfDlwIfAPwx0TEedFRFdEdD3yyCO1TVCSJEmS6lQRhWAvMHHAdkc5ti03AHOG25GZizKzMzM7J0yYMIopSpIkSVLjKqIQXAFMjohJEbE7cBawbOABETF5wOYpwL01zE+SJEmSGlrN7xHMzKcj4nxgOTAO+Hxm3h0RC4GuzFwGnB8RfwZsBh4H/qLWeUqSJElSoyqiWQyZeQtwy5DY/AHPP1TzpCRJkiSpSYzZZjGSJEmSpOqwEJQkSZKkJmMhKEmSJElNxkJQkiRJkpqMhaAkSZIkNZlCuoZKkiSp+WzevJmenh6eeOKJolOR6toee+xBR0cHLS0tu3wOC8EGtHRVL5ctX8tDG/vYv62VebOmMGdGe9FpSZKkJtfT08OLX/xiDjroICKi6HSkupSZPProo/T09DBp0qRdPo9TQxvM0lW9XLxkNb0b+0igd2MfFy9ZzdJVvUWnJkmSmtwTTzzB3nvvbREoVSAi2HvvvSseWbcQbDCXLV9L3+Ytg2J9m7dw2fK1BWUkSZL0LItAqXKj8d+RhWCDeWhj34jikiRJzeQv//IveelLX8rUqVO3eUxmcsEFF3DIIYcwffp0fvKTn2zdd+211zJ58mQmT57Mtddeu8Prffe73+XNb37zqOS+K6655hoeeuihwq4/2r7xjW8wZcoUDjnkEC655JJhj/nUpz7FYYcdxvTp0znhhBN48MEHt+4bN24cRxxxBEcccQSzZ8/e4fUeeOCB7X5Wqm3p0qWsWbOmKue2EKySpat6OfqS25l00dc5+pLbazY1c/+21hHFJUmSmsm5557LN77xje0ec+utt3Lvvfdy7733smjRIt7//vcD8Nhjj/Gxj32MO++8kx//+Md87GMf4/HHH9+lPJ5++uldet1INVIhuGXLFj7wgQ9w6623smbNGq6//vphi6QZM2bQ1dVFd3c3b3nLW/jIRz6ydV9rayt33XUXd911F8uWLdvlXGr1/bMQrDNF3qc3b9YUWlvGDYq1toxj3qwpVb+2JEnSaKrGH9aPPfZY9tprr+0e87WvfY13vvOdRASvec1r2LhxIw8//DDLly/nxBNPZK+99mLPPffkxBNPHLao/MY3vsGhhx7KkUceyZIlS7bGFyxYwDnnnMPRRx/NOeecwxNPPMG73vUupk2bxowZM/jOd74DlIq30047jeOOO47JkyfzsY99bOs5PvWpTzF16lSmTp3Kpz/9aeC5o1aXX345CxYs4KabbqKrq4u3v/3tHHHEEfT11XiGWPdiuGIqLGgrPXYvruh0P/7xjznkkEM4+OCD2X333TnrrLP42te+9pzj3vCGN/CCF7wAgNe85jX09PSM6DorV67k8MMP5/DDD+fqq6/eGr/mmmuYPXs2xx9/PCeccAKZybx585g6dSrTpk3jxhtvBEqjwMceeyynnHIKU6ZM4X3vex/PPPMMANdffz3Tpk1j6tSpXHjhhVvP/aIXvWjr85tuuolzzz2XH/7whyxbtox58+ZxxBFHcN99943ofeyIXUOrYHv36VW7e2f/+e0aKkmS6ln/H9b7f6fq/8M6UPXfa3p7e5k4ceLW7Y6ODnp7e7cZH+iJJ57gPe95D7fffjuHHHIIb33rWwftX7NmDXfccQetra388z//MxHB6tWr+fnPf84b3/hGfvGLXwCloudnP/sZL3jBCzjqqKM45ZRTiAi+8IUvcOedd5KZvPrVr+b1r389e+6557Dv4y1veQtXXXUVl19+OZ2dnaP1z7NzuhfDzRfA5nLxuWl9aRtg+txdOuVw//533nnndl/zuc99jpNPPnnr9hNPPEFnZye77bYbF110EXPmzHnOa971rndx1VVXceyxxzJv3rxB+37yk5/Q3d3NXnvtxX/8x39w11138dOf/pQNGzZw1FFHceyxxwKl79+aNWs48MADOemkk1iyZAmvfe1rufDCC1m5ciV77rknb3zjG1m6dOmwOQC89rWvZfbs2bz5zW/mLW95y07/O+0sC8EqKPo+vTkz2i38JElSXSvyD+uV+PnPf86kSZOYPHkyAO94xztYtGjR1v2zZ8+mtbV0y84dd9zBBz/4QQAOPfRQDjzwwK2F4Iknnsjee+8NwBlnnMEdd9xBRHD66afzwhe+cGv8v/7rv3bqXreau23hs0Vgv819pfguFoIj9aUvfYmuri6+973vbY09+OCDtLe3c//993P88cczbdo0Xv7yl2/dv3HjRjZu3Li1oDvnnHO49dZbt+7vHxGG0vfv7LPPZty4cey77768/vWvZ8WKFbzkJS9h5syZHHzwwQCcffbZ3HHHHbS0tHDccccxYcIEAN7+9rfz/e9/f5uFYLU5NbQK9m9rZfbz7uCO3S/g/ue/jTt2v4DZz7vD+/QkSZJ2UpF/WG9vb2f9+vVbt3t6emhvb99mfCT6i7gdGdoVcntdInfbbbetUw+BipcVGBWbtjEdc1vxnTCSf/9vf/vbfPzjH2fZsmU8//nPH3QOgIMPPpjjjjuOVatWjSiHanz/hu6v1ffPQrAKPn3YvXyy5bN0PG8DzwvoeN4GPtnyWT592L1FpyZJklQXimyAN3v2bK677joykx/96EeMHz+e/fbbj1mzZvHNb36Txx9/nMcff5xvfvObzJo1a9BrDz30UB544IGt93Ndf/3127zOMcccw5e//GUAfvGLX/CrX/2KKVNKfR2+9a1v8dhjj9HX18fSpUs5+uijOeaYY1i6dCn/8z//wx//+Ee++tWvcswxx7Dvvvvy29/+lkcffZQnn3yS//zP/9x6jRe/+MX8/ve/H+1/oh0b3zGy+E446qijuPfee/nlL3/JU089xQ033DDsaOiqVat473vfy7Jly3jpS1+6Nf7444/z5JNPArBhwwZ+8IMfcNhhhw16bVtbG21tbdxxxx0AW78/wznmmGO48cYb2bJlC4888gjf//73mTlzJlCaGvrLX/6SZ555hhtvvJHXve51zJw5k+9973ts2LCBLVu2cP311/P6178egH333Zd77rmHZ555hq9+9atbr1HN75+FYBUcdd+/0BpPDYq1xlMcdd+/FJSRJElSfalWA7yzzz6bP/3TP2Xt2rV0dHTwuc99DoDPfOYzfOYznwHgTW96EwcffDCHHHII73nPe/g//+f/ALDXXnvx0Y9+lKOOOoqjjjqK+fPnP6fxzB577MGiRYs45ZRTOPLIIwcVIkP99V//Nc888wzTpk3jrW99K9dcc83W0auZM2dy5plnMn36dM4880w6Ozs58sgjOffcc5k5cyavfvWrefe7382MGTNoaWlh/vz5zJw5kxNPPJFDDz106zXOPfdc3ve+99W+WcwJ86FlSNHe0lqK76LddtuNq666ilmzZvGKV7yCuXPn8spXvhKA+fPnb+0COm/ePP7whz/w53/+54OWibjnnnvo7Ozk8MMP5w1veAMXXXTRcwpBgC984Qt84AMf4IgjjiAzt5nP6aefzvTp0zn88MM5/vjjufTSS3nZy14GlIrW888/n1e84hVMmjSJ008/nf32249LLrmEN7zhDRx++OG86lWv4rTTTgPgkksu4c1vfjOvfe1r2W+//bZe46yzzuKyyy5jxowZo94sJrb35upJZ2dndnV1FZ1GyYI2YLh/14AFG2udjSRJ0phwzz338IpXvGKnj1+6qrcpG+Bdc801dHV1cdVVVxWdSmW6F5fuCdzUUxoJPGF+ze4PLNJ3v/tdLr/88kEjs9Uw3H9PEbEyM3eqM5DNYqphfEepM9JwcUmSJO0UG+DVuelzm6Lwq1eFTA2NiJMiYm1ErIuIi4bZ/7cRsSYiuiPitog4sIg8d1kVhsIlSZLUHM4999z6Hw1sYscdd1zVRwNHQ80LwYgYB1wNnAwcBpwdEUMn564COjNzOnATcGlts6zQ9Llw6pUwfiIQpcdTr/QvIpIkSZLGhCKmhs4E1mXm/QARcQNwGrCm/4DM/M6A438EvKOmGY4Gh8IlSZKeIzN32Epf0vaNRp+XIqaGtgMDb6DrKce25a+AW7ezX5IkSXVgjz324NFHHx2VX2KlZpWZPProo+yxxx4VnWdMN4uJiHcAncDrt7H/POA8gAMOOKCGmUmSJGmkOjo66Onp4ZFHHik6Famu7bHHHnR0VNaIsohCsBeYOGC7oxwbJCL+DPh/gNdn5pPDnSgzFwGLoLR8xOinKkmSpNHS0tLCpEmTik5DEsVMDV0BTI6ISRGxO3AWsGzgARExA/i/wOzM/G0BOUqSJElSw6p5IZiZTwPnA8uBe4DFmXl3RCyMiNnlwy4DXgT8e0TcFRHLtnE6SZIkSdIIFXKPYGbeAtwyJDZ/wPM/q3lSkiRJktQkolG6NkXEI8CDRecxjH2ADUUnoYbmZ0zV5OdL1eTnS9Xk50vVNFY/Xwdm5oSdObBhCsGxKiK6MrOz6DzUuPyMqZr8fKma/Hypmvx8qZoa4fNVRLMYSZIkSVKBLAQlSZIkqclYCFbfoqITUMPzM6Zq8vOlavLzpWry86VqqvvPl/cISpIkSVKTcURQkiRJkpqMhWAVRcRJEbE2ItZFxEVF56PGEhEPRMTqiLgrIrqKzkf1LyI+HxG/jYifDYjtFRHfioh7y497Fpmj6tc2Pl8LIqK3/HPsroh4U5E5qn5FxMSI+E5ErImIuyPiQ+W4P8NUse18vur6Z5hTQ6skIsYBvwBOBHqAFcDZmbmm0MTUMCLiAaAzM8fiGjaqQxFxLPAH4LrMnFqOXQo8lpmXlP+gtWdmXlhknqpP2/h8LQD+kJmXF5mb6l9E7Afsl5k/iYgXAyuBOcC5+DNMFdrO52sudfwzzBHB6pkJrMvM+zPzKeAG4LSCc5KkbcrM7wOPDQmfBlxbfn4tpf/xSSO2jc+XNCoy8+HM/En5+e+Be4B2/BmmUbCdz1ddsxCsnnZg/YDtHhrgA6MxJYFvRsTKiDiv6GTUsPbNzIfLz38N7FtkMmpI50dEd3nqqNP2VLGIOAiYAdyJP8M0yoZ8vqCOf4ZZCEr163WZeSRwMvCB8rQrqWqydC+B9xNoNP0r8HLgCOBh4J+LTUf1LiJeBPwH8DeZ+buB+/wZpkoN8/mq659hFoLV0wtMHLDdUY5JoyIze8uPvwW+Smk6sjTaflO+N6L/HonfFpyPGkhm/iYzt2TmM8C/4c8xVSAiWij9kv7lzFxSDvszTKNiuM9Xvf8MsxCsnhXA5IiYFBG7A2cBywrOSQ0iIl5YvlmZiHgh8EbgZ9t/lbRLlgF/UX7+F8DXCsxFDab/F/Sy0/HnmHZRRATwOeCezPzUgF3+DFPFtvX5qvefYXYNraJyC9lPA+OAz2fmxwtOSQ0iIg6mNAoIsBvwFT9fqlREXA8cB+wD/Ab438BSYDFwAPAgMDczbfihEdvG5+s4SlOqEngAeO+A+7mknRYRrwP+C1gNPFMO/z2l+7j8GaaKbOfzdTZ1/DPMQlCSJEmSmoxTQyVJkiSpyVgISpIkSVKTsRCUJEmSpCZjIShJkiRJTcZCUJIkSZKajIWgJElDRMSWiLhrwNdFo3jugyKirtaakiQ1nt2KTkCSpDGoLzOPKDoJSZKqxRFBSZJ2UkQ8EBGXRsTqiPhxRBxSjh8UEbdHRHdE3BYRB5Tj+0bEVyPip+Wv15ZPNS4i/i0i7o6Ib0ZEa2FvSpLUlCwEJUl6rtYhU0PfOmDfpsycBlwFfLoc+xfg2sycDnwZuLIcvxL4XmYeDhwJ3F2OTwauzsxXAhuBM6v8fiRJGiQys+gcJEkaUyLiD5n5omHiDwDHZ+b9EdEC/Doz946IDcB+mbm5HH84M/eJiEeAjsx8csA5DgK+lZmTy9sXAi2Z+Y/Vf2eSJJU4IihJ0sjkNp6PxJMDnm/Be/YlSTVmIShJ0si8dcDjf5ef/xA4q/z87cB/lZ/fBrwfICLGRcT4WiUpSdL2+BdISZKeqzUi7hqw/Y3M7F9CYs+I6KY0qnd2OfZB4AsRMQ94BHhXOf4hYFFE/BWlkb/3Aw9XPXtJknbAewQlSdpJ5XsEOzNzQ9G5SJJUCaeGSpIkSVKTcURQkiRJkpqMI4KSpKZSXvw9I8L75CVJTctCUJJUVyLiGxGxcJj4aRHxaws8SZJ2zEJQklRvrgXeERExJH4O8OXMfLqAnEZFlPj/ZklS1fk/G0lSvVkK7A0c0x+IiD2BNwPXlbdPiYhVEfG7iFgfEQt29uQRcVFE3BcRv4+INRFx+pD974mIewbsP7IcnxgRSyLikYh4NCKuKscXRMSXBrx+0NTUiPhuRHw8In4A/A9wcES8a8A17o+I9w7J4bSIuKv8/u6LiJMi4s8jYuWQ4/42Ir62s+9dktQ8LAQlSXUlM/uAxcA7B4TnAj/PzJ+Wt/9Y3t8GnAK8PyLm7OQl7qNUZI4HPgZ8KSL2A4iIPwcWlM/9EmA28GhEjAP+E3gQOAhoB24Ywds6BzgPeHH5HL+lVNi+hNKahFcMKDhnUip455Xf37HAA8AyYFJEvGLIea8bQR6SpCZhIShJqkfXAm+JiD3K2+8sxwDIzO9m5urMfCYzu4HrgdfvzIkz898z86Hya28E7gVmlne/G7g0M1dkybrMfLC8f39gXmb+MTOfyMw7RvB+rsnMuzPz6czcnJlfz8z7ytf4HvBNnh0B/Svg85n5rXKOvZn588x8ErgReAdARLySUlH6nyPIQ5LUJCwEJUl1p1xkbQDmRMTLKRViX+nfHxGvjojvlKdpbgLeB+yzM+eOiHeWp11ujIiNwNQBr51IacRwqInAgxXcn7h+SA4nR8SPIuKxcg5v2okcoFQMv618/+Q5wOJygShJ0iAWgpKkenUdpZHAdwDLM/M3A/Z9hdJUyYmZOR74DDC0ucxzRMSBwL8B5wN7Z2Yb8LMBr10PvHyYl64HDthGx9I/Ai8YsP2yYY7ZuqhvRDwf+A/gcmDfcg637EQOZOaPgKcojR6+DfjicMdJkmQhKEmqV9cBfwa8hwHTQsteDDyWmU+U76l7206e84WUirJHACLiXZRGBPt9FvhfEfGqcofPQ8rF44+Bh4FLIuKFEbFHRBxdfs1dwLERcUBEjAcu3kEOuwPPL+fwdEScDLxxwP7PAe+KiBMi4nkR0R4Rhw7Yfx1wFbB5hNNTJUlNxEJQklSXMvMB4IeUirdlQ3b/NbAwIn4PzKfUXGZnzrkG+Gfgv4HfANOAHwzY/+/AxymNOP6eUgfTvTJzC3AqcAjwK6AHeGv5Nd+idO9eN7CSHdyzl5m/By4o5/w4pSJ22YD9P6bcQAbYBHwPOHDAKb5IqXj9EpIkbUNk5o6PkiRJdSEiWil1HT0yM+8tOh9J0tjkiKAkSY3l/cAKi0BJ0vYMd1O7JEmqQxHxAKWmMju7ZqIkqUk5NVSSJEmSmoxTQyVJkiSpyVgISpIkSVKTaZh7BPfZZ5886KCDik5DkiRJkgqxcuXKDZk5YWeObZhC8KCDDqKrq6voNCRJkiSpEBHx4M4e69RQSZIkSWoyFoKSJEmS1GQsBCVJkiSpyVgISpIkSVKTsRCUJEmSpCZjIShJkiRJTcZCUJKkInUvhiumwoK20mP34qIzkiQ1gYZZR1CSpLrTvRhuvgA295W2N60vbQNMn1tcXmoc3YvhtoWwqQfGd8AJ8/1sSQIcEZQkqTi3LXy2COy3ua8UlyrV/4eGTeuBfPYPDY46S8JCUJKk4mzqGVlc9auIKcD+oaH2nOqtOuLUUEmSijK+ozxaM0xcjaOoKcD+oaG2nOqtOuOIoCRJRTlhPrS0Do61tJbiahxFjcxt6w8K/qGhOhyBVZ2paiEYESdFxNqIWBcRFw2z/30RsToi7oqIOyLisHL8xIhYWd63MiKOr2aekiQVYvpcOPVKGD8RiNLjqVc6etBoihqZ8w8NteUIrOpM1aaGRsQ44GrgRKAHWBERyzJzzYDDvpKZnykfPxv4FHASsAE4NTMfioipwHKgvVq5SpJUmOlzLfwaXVFTgPs/V3YNrQ2neqvOVPMewZnAusy8HyAibgBOA7YWgpn5uwHHvxDIcnzVgPjdQGtEPD8zn6xivpIkSaPvhPmD7x2D2o3M+YeG2iny+yztgmpODW0HBv5ZpIdhRvUi4gMRcR9wKXDBMOc5E/iJRaAkSapL0+eyYtrH+DUTeCaDXzOBFdM+ZoHWaJzqrTpTeNfQzLwauDoi3gb8A/AX/fsi4pXAJ4E3DvfaiDgPOA/ggAMOqH6ykiRJI7R0VS8XrziQvs3/39ZY64pxfGJiL3NmeOdLQ3EEtna6FzvtuULVHBHsBSYO2O4ox7blBmBO/0ZEdABfBd6ZmfcN94LMXJSZnZnZOWHChFFIWZIkaXRdtnwtfZu3DIr1bd7CZcvXFpSRVOf6l+rYtB7IZ5fqcN3GEalmIbgCmBwRkyJid+AsYNnAAyJi8oDNU4B7y/E24OvARZn5gyrmKEmSVFUPbewbUVzSDrhUx6ioWiGYmU8D51Pq+HkPsDgz746IheUOoQDnR8TdEXEX8Lc8Oy30fOAQYH55aYm7IuKl1cpVkiSpWvZvax1RXNIOuFTHqKjqPYKZeQtwy5DY/AHPP7SN1/0j8I/VzE2SJKkW5s2awsVLVg+aHtraMo55s6YUmJVUx1yqY1RUdUF5SZKkZjdnRjufOGMa7W2tBNDe1sonzphmoxhpV50wv7Q0x0Au1TFihXcNlSRJanRzZrQ3X+FnV0dVS//nyM9XRSwEJUmSNLr6uzr2N/To7+oI/rKu0eFSHRVzaqgkSZJGl10dpTHPQlCSJEmjy66O0phnIShJkqTRta3ujXZ1lMYMC0FJ0tjRvRiumAoL2kqP3YuLzqix+e+tarGrozTm2SxGkjQ22FyitroX8/TXPshuW54obW9aX9oG/71VObs6SmNeZGbROYyKzv+/vXuPs6quF///ejcOMWY5amYCXvArX5QAwQYszXuKpuIlQy0ru9lFj3V6HBJ+ncMxv9++csAuP492itNF+1YqGk50SsmjdqFzNEcxSIw08zKDpahY5igI7+8few8NOMAMzJo9e+/X8/GYx97rvdba6z3Mms1+z+e9PqulJdva2iqdhiRpW31p7GZuELwX/P1vBj6fGvfCvxzAjp1PvDLetCc7XvzbCmQkSdpeEXFPZrb0ZltbQyVJg4OTSwyooZ1/7FNcklRbLARVGyp1nYvX10j9x8klBtTK9bv1KS5pEPPziLaBhaCqX9d1Rc89DuTfrisq+k2wUseVapWTSwyorw85lxdyyEaxF3IIXx9yboUykrRN/DyibWQhqOpXqZvWerNcqX+NnwanXFG6JpAoPZ5yRc1PLtG6pIPDZt/OyBk/4rDZt9O6pGNAjjvhpPOZlefTvv71rM+gff3rmZXnM+Gk8wfk+JL6iZ9HtI2cNVTVr1LXFXk9k9T/xk+r+cKvu9YlHcxcsIzOtesA6FjdycwFywA4beLwQo9dev1PcNaiY1m5upNhzU1MnzK68ONK6md+HtE2shBU9dt5xGZmGiz4uqJKHVdSzZi7aMWGIrBL59p1zF20YkAKstMmDrfwk6qdn0e0jWwNVfWr1HVFXs8kaTutXN3Zp7gkvYKfR7SNHBFU9avUTWu9Wa4K1rqkg7mLVti2V8OGNTfR0UPRN6y5qYetJakHfh7RNvKG8pI0CG167RhAU2MDl50xzmKwhvhzliT1p77cUN4RQUkahOYuWsFx637GZ4bMZ1isYmW+njkvT2PuoiEWCDWk62fpyK8kaaBZCErSINTy51u5rPHr7BhrABgRq5jd+HVm/hngmIrmpv7lhC1SP1s63zZJqRecLEaSBqGZQ27YUAR22THWMHPIDRXKSJKqgDdXl3qt0EIwIk6IiBUR8VBEzOhh/cciYllE3BcRiyNiTLd1M8v7rYiIKUXmKUmDzR6s6lNckoQ3V5f6oLBCMCIagKuAE4ExwDndC72y72XmuMycAMwBvljedwxwNvAm4ATgK+XXk6S6EJu5/9Pm4v1u6Xz40li4pLn06F/T1Z88v1QUb64u9VqRI4KTgYcy8+HMXANcB5zafYPM/HO3xdcAXVOYngpcl5kvZeYfgIfKrydJ9aGS94WytUpF8vxSkTb3xzJvri69QpGF4HDg8W7L7eXYRiLigoj4PaURwYv6sq8k1azx0+CUK2DnvYAoPZ5yxcBMeGBrlYrk+aUieXN1qdcqPmtoZl4FXBUR7wb+EXh/b/eNiPOB8wH23nvvYhKUpEoZP60yM93ZWqUiVfL8cjbJ2ufN1Qeev1dVq8hCsAPYq9vyiHJsc64D/q0v+2bmPGAelG4ovz3JSuol3/Br384jym17PcSl7fRC0xvZsfOJnuNFHrirJbVrNLKrJRV8D6s1lfojWj3y96qqFdkaejcwKiJGRsQQSpO/LOy+QUSM6rZ4EvBg+flC4OyIeHVEjARGAb8qMFdJveG1PfXB1ioVaM7as3ghh2wUeyGHMGftWcUe2JZUqf/5e1XVChsRzMyXI+JCYBHQAHwzM++PiEuBtsxcCFwYEW8H1gLPUm4LLW83H1gOvAxckJnrispVUi9t6Q3fv/zVDlurVKBrnp/MM69aw2d2mM+weJqVuXIpyhsAACAASURBVBtzXp7GD1+azCVFHtiWZw2A1iUdzF20gpWrOxnW3MT0KaM5bWINT3NRh79XtfQzLvQawcz8MfDjTWKzuj3/5Bb2/Tzw+eKyk9RndfiGX7dsrVJBhjU3sXD121i45m0bxYc3N21mj35iy3PdqNQH9dYlHcxcsIzOtaWxi47VncxcsAygaguFraqz36ta+xkXekN5STXGabklbafpU0bT1LjxrYGbGhuYPmV0sQe25bkudH1Q71jdSfK3D+qtS7Y0TUX/mLtoxYYCoUvn2nXMXbSi8GNXTJ39XtXaz9hCUFLv1dkbvqT+d9rE4Vx2xjiGNzcRlEYCLztjXPF/Ta/kLVk0YCr5QX3l6s4+xWtCnf1e1drPuOK3j5BURSp97Zgzlko14bSJwyvTRmXLc82r5Af1Yc1NdPRwnGFFtz1XWoV+ryrRAjysuYk3//nW8jXOq1iZr2fOy9O453XHFXrcolgISuqbSn2QcopqSdJWVLIYmz5l9EbXj8EAtT3XoUpdq/flMQ8y9p6v0xRrABgRq/iXxq/zmzH7AscUdtyi2BoqqTo4RbUkaSsqdg0qFWx7rkOVagGe9Pt/3VAEdmmKNUz6/b8WetyiOCIoVat6a5N0xlJJ0lZ0FV2Vmt6/Ym3PdaZiLcA19lnEQlCqRvXYJllnU1RLkraNxVjtq1gLcI19FrE1VKpG9dgm6YylktRnrUs6OGz27Yyc8SMOm337gNxGQSqat6HpH44IStWoxloTeqXSM5ZKUpWptZtfa/MqMYNmJVWsBbjGPotEZlY6h37R0tKSbW1tlU5DGhhfGruZ1oS94O9/M/D5SJIGncNm395j+9zw5iZ+OaP6ZjhUzzYt+KE0OuZENfUpIu7JzJbebGtrqFSNaqw1QZLU/2rt5tfqWaVm0FT1szVUqkY11pogDQr1NhOval7d3uC8zlS64K+3ttRaYiEoVatK3dhdqkX1OBOvap43OK8PlSz4vQ61utkaKklSPc7Eq5rnDc7rQ8Vm0MS21GrniKAkqe7lc+1EH+KqTvXYwuY99WpfxWbQpPJtqdo+FoKSpLr3J17PG3lqM3HVAlvYVMsqVfB7HWp1szVUklT3LlvzLl7IIRvFXsghXLbmXRXKSP3NFjap/1WyLVXbzxFBqUrVY4uTal+lzuu21x3HjD/DZ3aYz7B4mpW5G3NensY9rzuu8GNrYNjCJvW/SralavtZCEpVyBYn1aJKntel2RXXsHDN2zbEmhobuMy/atcMW9ikYngdavWyNVSqQrY4DazWJR0cNvt2Rs74EYfNvp3WJR2VTqkmVfK8dnbF2mcLmyRtzBFBqQrZ4jRwHH0dOJU+r/2rdm2zhW3geQmDNLgVWghGxAnA/w80AF/PzNmbrP808GHgZeAp4IOZ+Wh53RzgJEqjlrcCn8zMLDJfqVrY4jRwtjRK5Qea/uV5raJZ7A8c/4gmDX6FtYZGRANwFXAiMAY4JyLGbLLZEqAlM8cDNwJzyvseChwGjAfGApOAI4vKVao2tjgNnEqPUtUTz2updngJgzT4FXmN4GTgocx8ODPXANcBp3bfIDPvyMwXyot3AiO6VgFDgSHAq4FG4E8F5ipVFa9nGjibG41ylKr/eV5LtcM/okmDX5GtocOBx7sttwOHbGH7DwE3A2Tmf0fEHcATQABXZuYDRSUqVSNbnAZGaTbJZRv9ZdtRquJ4Xku1wVZvafAbFLOGRsS5QAswt7y8P3AgpRHC4cAxEXF4D/udHxFtEdH21FNPDWTKkuqEo1SS1He2ekuDX5Ejgh3AXt2WR5RjG4mItwOfBY7MzJfK4dOBOzPz+fI2NwNvBX7Rfd/MnAfMA2hpaXEiGUmFcJRKkvrGWVqlwa/IQvBuYFREjKRUAJ4NvLv7BhExEfgacEJmPtlt1WPARyLiMkqtoUcCXy4wV0mSJPUj/4gmDW6FtYZm5svAhcAi4AFgfmbeHxGXRsTU8mZzgZ2AGyLivohYWI7fCPweWAb8Gvh1Zv6wqFwlSZIkqZ5Erdyar6WlJdva2iqdhiRJkiRVRETck5ktvdl2UEwWI0mSJEkaOBaCkiRJklRnLAQlSZIkqc5YCEqSJElSnbEQlCRJkqQ6U+R9BCWpNiydD7ddCs+1w84j4NhZMH5apbMqVOuSDm8ELUlSDbMQlKQtWToffngRrO0sLT/3eGkZarYYbF3SwcwFy+hcuw6AjtWdzFywDMBiUJKkGmFrqCRtyW2X/q0I7LK2sxSvUXMXrdhQBHbpXLuOuYtWVCgjSZLU3ywEJWlLnmvvW7wGrFzd2ae4JEmqPhaCkrQlO4/oW7wGDGtu6lNckiRVHwtBSdqSY2dB4yYFUGNTKV6jpk8ZTVNjw0axpsYGpk8ZXaGMJElSf3OyGEnakq4JYepo1tCuCWGcNVSSpNoVmVnpHPpFS0tLtrW1VToNSZIkSaqIiLgnM1t6s62toZIkSZJUZywEJUmSJKnOWAhKkiRJUp3ZaiEYEX8XEbsMRDKSJEmSpOL1ZtbQPYC7I+Je4JvAoqyVGWYkVZXWJR3OZClJktQPtjoimJn/CIwCvgGcBzwYEf8nIv5HwblJ0gatSzqYuWAZHas7SaBjdSczFyyjdUlHpVOTJEmqOr26RrA8AvjH8tfLwC7AjRExp8DcJGmDuYtW0Ll23UaxzrXrmLtoRYUykiRJql69uUbwkxFxDzAH+CUwLjM/DrwZeOdW9j0hIlZExEMRMaOH9Z+OiOURsTQibouIfbqt2zsifhIRD5S32beP35ukGrJydWef4pIkSdq83lwjuCtwRmY+2j2Ymesj4uTN7RQRDcBVwHFAO6XrDBdm5vJumy0BWjLzhYj4OKVi86zyum8Dn8/MWyNiJ2B9r78rSTVnWHMTHT0UfcOamyqQjSRJUnXrTWvozcAzXQsR8bqIOAQgMx/Ywn6TgYcy8+HMXANcB5zafYPMvCMzXygv3gmMKB9jDLBDZt5a3u75bttJqkPTp4ymqbFho1hTYwPTp4yuUEaSJEnVqzeF4L8Bz3dbfr4c25rhwOPdltvLsc35EKWiE+B/AqsjYkFELImIueURRkl16rSJw7nsjHEMb24igOHNTVx2xjhnDZUkSdoGvWkNje63iyi3hPZmv16LiHOBFuDIbnkdDkwEHgOupzRj6Tc22e984HyAvffeuz9TkjQInTZxuIWfJElSP+jNiODDEXFRRDSWvz4JPNyL/TqAvbotjyjHNhIRbwc+C0zNzJfK4XbgvnJb6ctAK3Dwpvtm5rzMbMnMlt13370XKUmSJEmSelMIfgw4lFIR1w4cQnkUbivuBkZFxMiIGAKcDSzsvkFETAS+RqkIfHKTfZsjoqu6OwboPsmMJEmSJGkbbbXFs1ygnd3XF87MlyPiQmAR0AB8MzPvj4hLgbbMXAjMBXYCbogIgMcyc2pmrouIfwBui9KKe4B/72sOkiRJkqRXim6X//W8QcRQShO5vAkY2hXPzA8Wm1rftLS0ZFtbW6XTkCRJkqSKiIh7MrOlN9v2pjX0/wJvBKYAP6N0rd9ftj09SZIkSVIl9aYQ3D8z/wn4a2ZeA5xE6TpBSZIkSVIV6k0huLb8uDoixgI7A28oLiVJkiRJUpF6cz/AeRGxC/CPlGb93An4p0KzkjRotS7pYO6iFaxc3cmw5iamTxntvf0kSZKqzBYLwYh4FfDnzHwW+Dmw34BkJWlQal3SwcwFy+hcuw6AjtWdzFywDMBiUJIkqYpssTU0M9cDnxmgXCQNcnMXrdhQBHbpXLuOuYtWVCgjSZIkbYveXCP4nxHxDxGxV0Ts2vVVeGaSBp2Vqzv7FJckSdLg1JtrBM8qP17QLZbYJirVnWHNTXT0UPQNa26qQDaSJEnaVlsdEczMkT18WQRKdWj6lNE0NTZsFGtqbGD6lNEVykiSJEnbYqsjghHxvp7imfnt/k9H0mDWNSGMs4ZKkiRVt960hk7q9nwocCxwL2AhKNWh0yYOt/CTJEmqclstBDPz77ovR0QzcF1hGUmSJEmSCtWbWUM39VdgZH8nIkmSJEkaGL25RvCHlGYJhVLhOAaYX2RSkiRJkqTi9OYawcu7PX8ZeDQz2wvKR5IkSZJUsN4Ugo8BT2TmiwAR0RQR+2bmI4VmJkmSJEkqRG8KwRuAQ7stryvHJvW8uVQ/Wpd0eCsFSZIkVZ3eFII7ZOaaroXMXBMRQwrMSaoKrUs6mLlgGZ1r1wHQsbqTmQuWAVgMSpIkaVDrzayhT0XE1K6FiDgVWFVcSlJ1mLtoxYYisEvn2nXMXbSiQhlJkiRJvdObEcGPAd+NiCvLy+3A+4pLSeq7SrRorlzd2ae4JEmSNFj05obyvwfeEhE7lZefLzwrqQ8q1aI5rLmJjh6KvmHNTYUdU5IkSeoPW20NjYj/ExHNmfl8Zj4fEbtExP/uzYtHxAkRsSIiHoqIGT2s/3RELI+IpRFxW0Tss8n610VEe7fRSOkVKtWiOX3KaJoaGzaKNTU2MH3K6EKPK0mSJG2v3lwjeGJmru5ayMxngXdsbaeIaACuAk6kdBP6cyJizCabLQFaMnM8cCMwZ5P1/wv4eS9yVB2rVIvmaROHc9kZ4xje3EQAw5ubuOyMcU4UI0mSpEGvN9cINkTEqzPzJSjdRxB4dS/2mww8lJkPl/e7DjgVWN61QWbe0W37O4FzuxYi4s3AHsAtQEsvjqc6VckWzdMmDrfwkyRJUtXpzYjgd4HbIuJDEfFh4Fbgml7sNxx4vNtyezm2OR8CbgaIiFcBXwD+oRfHUZ2zRVOSJEnqm95MFvMvEfFr4O1AAouAfba8V99ExLmURv2OLIc+Afw4M9sjYkv7nQ+cD7D33nv3Z0qqIl0jct7YXZIkSeqd3rSGAvyJUhH4LuAPwPd7sU8HsFe35RHl2EYi4u3AZ4Eju9pPgbcCh0fEJ4CdgCER8XxmbjThTGbOA+YBtLS0ZC+/F9UgWzQlSZKk3ttsIRgR/xM4p/y1CrgeiMw8upevfTcwKiJGUioAzwbevckxJgJfA07IzCe74pn5nm7bnEdpQplXzDoqSZIkSeq7LY0I/hb4BXByZj4EEBF/39sXzsyXI+JCSq2kDcA3M/P+iLgUaMvMhcBcSiN+N5RbQB/LzKnb9q1IkiRJknpjS4XgGZRG8e6IiFuA64DNX7DXg8z8MfDjTWKzuj1/ey9e42rg6r4cV5IkSZK0eZudNTQzWzPzbOAA4A7gU8AbIuLfIuL4gUpQkiRJktS/tnr7iMz8a2Z+LzNPoTThyxLg4sIzkyRJkiQVojf3EdwgM5/NzHmZeWxRCUmSJEmSitWnQlCSJEmSVP0sBCVJkiSpzlgISpIkSVKdsRCUJEmSpDpjIShJkiRJdcZCUJIkSZLqjIWgJEmSJNUZC0FJkiRJqjMWgpIkSZJUZywEJUmSJKnOWAiq/yydD18aC5c0lx6Xzq90RpIkSZJ6sEOlE1CNWDoffngRrO0sLT/3eGkZYPy0yuUlSZIk6RUcEVT/uO3SvxWBXdZ2luKSJEmSBhULQfWP59r7FpckSZJUMRaC6h87j+hbXJIkSVLFWAiqfxw7CxqbNo41NpXikiRJkgYVC0H1j/HT4JQrYOe9gCg9nnKFE8VIkiRJg5Czhqr/jJ9m4SdJkiRVgUJHBCPihIhYEREPRcSMHtZ/OiKWR8TSiLgtIvYpxydExH9HxP3ldWcVmackSZIk1ZPCCsGIaACuAk4ExgDnRMSYTTZbArRk5njgRmBOOf4C8L7MfBNwAvDliGguKldJkiRJqidFjghOBh7KzIczcw1wHXBq9w0y847MfKG8eCcwohz/XWY+WH6+EngS2L3AXCVJkiSpbhRZCA4HHu+23F6Obc6HgJs3DUbEZGAI8Pt+zU6SJEmS6tSgmCwmIs4FWoAjN4nvCfxf4P2Zub6H/c4HzgfYe++9ByBTSZIkSap+RRaCHcBe3ZZHlGMbiYi3A58FjszMl7rFXwf8CPhsZt7Z0wEycx4wD6ClpSX7L3Vti9YlHcxdtIKVqzsZ1tzE9CmjOW3ilgaBJUmSJFVCkYXg3cCoiBhJqQA8G3h39w0iYiLwNeCEzHyyW3wIcBPw7cy8scAc1U9al3Qwc8EyOteuA6BjdSczFywDsBiUJEmSBpnCrhHMzJeBC4FFwAPA/My8PyIujYip5c3mAjsBN0TEfRGxsByfBhwBnFeO3xcRE4rKVdtv7qIVG4rALp1r1zF30YoKZSRJkiRpcwq9RjAzfwz8eJPYrG7P376Z/b4DfKfI3NS/Vq7u7FNckiRJUuUUekN51Y9hzU19ikuSJEmqHAtB9YvpU0bT1NiwUaypsYHpU0ZXKCNJkiRJmzMobh+h6tc1IYyzhkqSJEmDn4Wg+s1pE4db+EmSJElVwNZQSZIkSaozFoKSJEmSVGcsBGvR0vnwpbFwSXPpcen8SmckSZIkaRDxGsFas3Q+/PAiWFu+f99zj5eWAcZPq1xekiRJkgYNRwRrzW2X/q0I7LK2sxSXJEmSJCwEa89z7X2LS5IkSao7FoK1ZucRfYtLkiRJqjsWgrXm2FnQ2LRxrLGpFJckSZIkLARrz/hpcMoVsPNeQJQeT7nCiWIkSZIkbeCsobVo/DQLP0mSJEmb5YigJEmSJNUZC0FJkiRJqjMWgpIkSZJUZywEJUmSJKnOWAhKkiRJUp2xEJQkSZKkOlNoIRgRJ0TEioh4KCJm9LD+0xGxPCKWRsRtEbFPt3Xvj4gHy1/vLzJPSZIkSaonhRWCEdEAXAWcCIwBzomIMZtstgRoyczxwI3AnPK+uwL/DBwCTAb+OSJ2KSpXSZIkSaonRY4ITgYeysyHM3MNcB1wavcNMvOOzHyhvHgnMKL8fApwa2Y+k5nPArcCJxSYqyRJkiTVjSILweHA492W28uxzfkQcPM27itJkiRJ6qUdKp0AQEScC7QAR/Zxv/OB8wH23nvvAjKTJEmSpNpT5IhgB7BXt+UR5dhGIuLtwGeBqZn5Ul/2zcx5mdmSmS277757vyUuSZIkSbWsyELwbmBURIyMiCHA2cDC7htExETga5SKwCe7rVoEHB8Ru5QniTm+HJMkSZIkbafCWkMz8+WIuJBSAdcAfDMz74+IS4G2zFwIzAV2Am6ICIDHMnNqZj4TEf+LUjEJcGlmPlNUrpIkSZJUTyIzK51Dv2hpacm2trZKpyFJkiRJFRER92RmS2+2LfSG8pIkSZKkwcdCUJIkSZLqjIWgJEmSJNUZC0FJkiRJqjMWgpIkSZJUZywEJUmSJKnOWAhKkiRJUp0p7Iby9a51SQdzF61g5epOhjU3MX3KaE6bOLzSaUmSJEmShWARWpd0MHPBMjrXrgOgY3UnMxcsA7AYlCRJklRxtoYWYO6iFRy37mcsHnIRD7/63SwechHHrfsZcxetqHRqkiRJkmQhWISWP9/K7MavM+JVq3hVwIhXrWJ249dp+fOtlU5NkiRJkiwEizBzyA3sGGs2iu0Ya5g55IYKZSRJkiRJf2MhWIA9WNWnuCRJkiQNJAvBAsTOI/oUlyRJkqSBZCFYhGNnQWPTxrHGplJckiRJkirMQrAI46fBKVfAznsBUXo85YpSXJIkSZIqzPsIFmX8NAs/SZKkbtauXUt7ezsvvvhipVORqtrQoUMZMWIEjY2N2/waFoKSJEkaEO3t7bz2ta9l3333JSIqnY5UlTKTp59+mvb2dkaOHLnNr2NrqCRJkgbEiy++yG677WYRKG2HiGC33Xbb7pF1C0FJkiQNGItAafv1x++RhaAkSZLqxgc/+EHe8IY3MHbs2M1uk5lcdNFF7L///owfP5577713w7prrrmGUaNGMWrUKK655pqtHu+nP/0pJ598cr/kvi2uvvpqVq5cWbHj97dbbrmF0aNHs//++zN79uwet/niF7/ImDFjGD9+PMceeyyPPvrohnUNDQ1MmDCBCRMmMHXq1K0e75FHHtniuVK01tZWli9fXshrF1oIRsQJEbEiIh6KiBk9rD8iIu6NiJcj4sxN1s2JiPsj4oGIuCL885EkSVJdaV3SwWGzb2fkjB9x2OzbaV3Ssd2ved5553HLLbdscZubb76ZBx98kAcffJB58+bx8Y9/HIBnnnmGz33uc9x111386le/4nOf+xzPPvvsNuXx8ssvb9N+fVXRQnDpfPjSWLikufS4dP52vdy6deu44IILuPnmm1m+fDnXXnttj0XSxIkTaWtrY+nSpZx55pl85jOf2bCuqamJ++67j/vuu4+FCxducy4D9fOrykIwIhqAq4ATgTHAORExZpPNHgPOA763yb6HAocB44GxwCTgyKJylSRJ0uDSuqSDmQuW0bG6kwQ6Vncyc8Gy7S4GjzjiCHbdddctbvODH/yA973vfUQEb3nLW1i9ejVPPPEEixYt4rjjjmPXXXdll1124bjjjuuxqLzllls44IADOPjgg1mwYMGG+CWXXMJ73/teDjvsMN773vfy4osv8oEPfIBx48YxceJE7rjjDqBUvJ166qkcddRRjBo1is997nMbXuOLX/wiY8eOZezYsXz5y18GXjlqdfnll3PJJZdw44030tbWxnve8x4mTJhAZ2fndv3b9cnS+fDDi+C5x4EsPf7wou0qBn/1q1+x//77s99++zFkyBDOPvtsfvCDH7xiu6OPPpodd9wRgLe85S20t7f36Tj33HMPBx10EAcddBBXXXXVhvjVV1/N1KlTOeaYYzj22GPJTKZPn87YsWMZN24c119/PVAaBT7iiCM46aSTGD16NB/72MdYv349ANdeey3jxo1j7NixXHzxxRtee6eddtrw/MYbb+S8887jv/7rv1i4cCHTp09nwoQJ/P73v+/T97E1Rc4aOhl4KDMfBoiI64BTgQ0lbWY+Ul63fpN9ExgKDAECaAT+VGCukiRJGkTmLlpB59p1G8U6165j7qIVnDZxeKHH7ujoYK+99tqwPGLECDo6OjYb7+7FF1/kIx/5CLfffjv7778/Z5111kbrly9fzuLFi2lqauILX/gCEcGyZcv47W9/y/HHH8/vfvc7oFT0/OY3v2HHHXdk0qRJnHTSSUQE3/rWt7jrrrvITA455BCOPPJIdtlllx6/jzPPPJMrr7ySyy+/nJaWlv765+md2y6FtZsUnms7S/FtvMVaT//+d9111xb3+cY3vsGJJ564YfnFF1+kpaWFHXbYgRkzZnDaaae9Yp8PfOADXHnllRxxxBFMnz59o3X33nsvS5cuZdddd+X73/8+9913H7/+9a9ZtWoVkyZN4ogjjgBKP7/ly5ezzz77cMIJJ7BgwQIOPfRQLr74Yu655x522WUXjj/+eFpbW3vMAeDQQw9l6tSpnHzyyZx55pk9brM9imwNHQ483m25vRzbqsz8b+AO4Iny16LMfKDfM5QkSdKgtHJ1z6NXm4sPFr/97W8ZOXIko0aNIiI499xzN1o/depUmpqaAFi8ePGG9QcccAD77LPPhkLwuOOOY7fddqOpqYkzzjiDxYsXs3jxYk4//XRe85rXsNNOO3HGGWfwi1/8YmC/wd56bjOjcJuLF+A73/kObW1tGxVzjz76KG1tbXzve9/jU5/61CtG2VavXs3q1as3FHTvfe97N1rfNSIMpZ/fOeecQ0NDA3vssQdHHnkkd999NwCTJ09mv/32o6GhgXPOOYfFixdz9913c9RRR7H77ruzww478J73vIef//znRf4TbNGgnCwmIvYHDgRGUCoej4mIw3vY7vyIaIuItqeeemqg0xy0iuinlyRJGkjDmpv6FO9Pw4cP5/HH/zae0d7ezvDhwzcb74vXvOY1vdpu0+kxtjRdxg477LCh9RDY7tsK9IudR/Qt3gt9+ff/z//8Tz7/+c+zcOFCXv3qV2/0GgD77bcfRx11FEuWLOlTDkX8/DZdP1A/vyILwQ5gr27LI8qx3jgduDMzn8/M54GbgbduulFmzsvMlsxs2X333bc74VpQVD+9JEnSQJo+ZTRNjQ0bxZoaG5g+ZXThx546dSrf/va3yUzuvPNOdt55Z/bcc0+mTJnCT37yE5599lmeffZZfvKTnzBlypSN9j3ggAN45JFHNow0XXvttZs9zuGHH853v/tdAH73u9/x2GOPMXp06fu79dZbeeaZZ+js7KS1tZXDDjuMww8/nNbWVl544QX++te/ctNNN3H44Yezxx578OSTT/L000/z0ksv8R//8R8bjvHa176Wv/zlL/39T7R1x86Cxk2K9samUnwbTZo0iQcffJA//OEPrFmzhuuuu67HmT+XLFnCRz/6URYuXMgb3vCGDfFnn32Wl156CYBVq1bxy1/+kjFjNp7CpLm5mebmZhYvXgyw4efTk8MPP5zrr7+edevW8dRTT/Hzn/+cyZMnA6XW0D/84Q+sX7+e66+/nre97W1MnjyZn/3sZ6xatYp169Zx7bXXcuSRpWlQ9thjDx544AHWr1/PTTfdtOEYRf78iiwE7wZGRcTIiBgCnA30dmqex4AjI2KHiGikNFGMraG9sKV+ekmSpGpx2sThXHbGOIY3NxHA8OYmLjtj3HZfH3jOOefw1re+lRUrVjBixAi+8Y1vAPDVr36Vr371qwC84x3vYL/99mP//ffnIx/5CF/5ylcA2HXXXfmnf/onJk2axKRJk5g1a9YrJp4ZOnQo8+bN46STTuLggw/eqBDZ1Cc+8QnWr1/PuHHjOOuss7j66qs3jF5NnjyZd77znYwfP553vvOdtLS0cPDBB3PeeecxefJkDjnkED784Q8zceJEGhsbmTVrFpMnT+a4447jgAMO2HCM8847j4997GMDP1nM+GlwyhWw815AlB5PuWKbrw+E0sjnlVdeyZQpUzjwwAOZNm0ab3rTmwCYNWvWhllAp0+fzvPPP8+73vWujW4T8cADD9DS0sJBBx3E0UcfzYwZM15RCAJ861vf4oILlBMnkwAABxpJREFULmDChAlk5mbzOf300xk/fjwHHXQQxxxzDHPmzOGNb3wjUCpaL7zwQg488EBGjhzJ6aefzp577sns2bM5+uijOeigg3jzm9/MqaeeCsDs2bM5+eSTOfTQQ9lzzz03HOPss89m7ty5TJw4sd8ni4ktfXPb/eIR7wC+DDQA38zMz0fEpUBbZi6MiEnATcAuwIvAHzPzTeUZR78CHEFp4phbMvPTWzpWS0tLtrW1Ffa9VIuRM35ETz/RAP4w+6SBTkeSJGmDBx54gAMPPLDSaQx6V199NW1tbVx55ZWVTkXb4Kc//SmXX375RiOzRejp9yki7snMXs0MVOSsoWTmj4EfbxKb1e353ZRaRjfdbx3w0SJzq1XDmpvo6OEi6oHop5ckSZJUHQblZDHadpXsp5ckSdL2O++88xwNrGJHHXVU4aOB/aHQEUENvK6++bmLVrBydSfDmpuYPmV04ffbkSRJklQ9LARr0GkTh1v4SZKkQSkztzqVvqQt6495XmwNlSRJ0oAYOnQoTz/9dL98iJXqVWby9NNPM3To0O16HUcEJUmSNCBGjBhBe3s7Tz31VKVTkara0KFDGTHiFXNu9omFoCRJkgZEY2MjI0eOrHQakrA1VJIkSZLqjoWgJEmSJNUZC0FJkiRJqjNRK7M2RcRTwKOVzqMHrwdWVToJ1TTPMRXJ80tF8vxSkTy/VKTBen7tk5m792bDmikEB6uIaMvMlkrnodrlOaYieX6pSJ5fKpLnl4pUC+eXraGSJEmSVGcsBCVJkiSpzlgIFm9epRNQzfMcU5E8v1Qkzy8VyfNLRar688trBCVJkiSpzjgiKEmSJEl1xkKwQBFxQkSsiIiHImJGpfNRbYmIRyJiWUTcFxFtlc5H1S8ivhkRT0bEb7rFdo2IWyPiwfLjLpXMUdVrM+fXJRHRUX4fuy8i3lHJHFW9ImKviLgjIpZHxP0R8cly3PcwbbctnF9V/R5ma2hBIqIB+B1wHNAO3A2ck5nLK5qYakZEPAK0ZOZgvIeNqlBEHAE8D3w7M8eWY3OAZzJzdvkPWrtk5sWVzFPVaTPn1yXA85l5eSVzU/WLiD2BPTPz3oh4LXAPcBpwHr6HaTtt4fyaRhW/hzkiWJzJwEOZ+XBmrgGuA06tcE6StFmZ+XPgmU3CpwLXlJ9fQ+k/PqnPNnN+Sf0iM5/IzHvLz/8CPAAMx/cw9YMtnF9VzUKwOMOBx7stt1MDJ4wGlQR+EhH3RMT5lU5GNWuPzHyi/PyPwB6VTEY16cKIWFpuHbVtT9stIvYFJgJ34XuY+tkm5xdU8XuYhaBUvd6WmQcDJwIXlNuupMJk6VoCrydQf/o34H8AE4AngC9UNh1Vu4jYCfg+8KnM/HP3db6HaXv1cH5V9XuYhWBxOoC9ui2PKMekfpGZHeXHJ4GbKLUjS/3tT+VrI7qukXiywvmohmTmnzJzXWauB/4d38e0HSKikdKH9O9m5oJy2Pcw9Yuezq9qfw+zECzO3cCoiBgZEUOAs4GFFc5JNSIiXlO+WJmIeA1wPPCbLe8lbZOFwPvLz98P/KCCuajGdH1ALzsd38e0jSIigG8AD2TmF7ut8j1M221z51e1v4c5a2iBylPIfhloAL6ZmZ+vcEqqERGxH6VRQIAdgO95fml7RcS1wFHA64E/Af8MtALzgb2BR4FpmemEH+qzzZxfR1FqqUrgEeCj3a7nknotIt4G/AJYBqwvh/8/Stdx+R6m7bKF8+scqvg9zEJQkiRJkuqMraGSJEmSVGcsBCVJkiSpzlgISpIkSVKdsRCUJEmSpDpjIShJkiRJdcZCUJKkTUTEuoi4r9vXjH587X0joqruNSVJqj07VDoBSZIGoc7MnFDpJCRJKoojgpIk9VJEPBIRcyJiWUT8KiL2L8f3jYjbI2JpRNwWEXuX43tExE0R8evy16Hll2qIiH+PiPsj4icR0VSxb0qSVJcsBCVJeqWmTVpDz+q27rnMHAdcCXy5HPtX4JrMHA98F7iiHL8C+FlmHgQcDNxfjo8CrsrMNwGrgXcW/P1IkrSRyMxK5yBJ0qASEc9n5k49xB8BjsnMhyOiEfhjZu4WEauAPTNzbTn+RGa+PiKeAkZk5kvdXmNf4NbMHFVevhhozMz/Xfx3JklSiSOCkiT1TW7meV+81O35OrxmX5I0wCwEJUnqm7O6Pf53+fl/AWeXn78H+EX5+W3AxwEioiEidh6oJCVJ2hL/AilJ0is1RcR93ZZvycyuW0jsEhFLKY3qnVOO/R3wrYiYDjwFfKAc/yQwLyI+RGnk7+PAE4VnL0nSVniNoCRJvVS+RrAlM1dVOhdJkraHraGSJEmSVGccEZQkSZKkOuOIoCRJkiTVGQtBSZIkSaozFoKSJEmSVGcsBCVJkiSpzlgISpIkSVKdsRCUJEmSpDrz/wADjD85NXC2mQAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 1080x1080 with 2 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Plot train and validation accuracies of the two models\n",
    "\n",
    "train_accs = []\n",
    "val_accs = []\n",
    "for dropout in dropout_choices:\n",
    "  solver = solvers[dropout]\n",
    "  train_accs.append(solver.train_acc_history[-1])\n",
    "  val_accs.append(solver.val_acc_history[-1])\n",
    "\n",
    "plt.subplot(3, 1, 1)\n",
    "for dropout in dropout_choices:\n",
    "  plt.plot(solvers[dropout].train_acc_history, 'o', label='%.2f dropout' % dropout)\n",
    "plt.title('Train accuracy')\n",
    "plt.xlabel('Epoch')\n",
    "plt.ylabel('Accuracy')\n",
    "plt.legend(ncol=2, loc='lower right')\n",
    "  \n",
    "plt.subplot(3, 1, 2)\n",
    "for dropout in dropout_choices:\n",
    "  plt.plot(solvers[dropout].val_acc_history, 'o', label='%.2f dropout' % dropout)\n",
    "plt.title('Val accuracy')\n",
    "plt.xlabel('Epoch')\n",
    "plt.ylabel('Accuracy')\n",
    "plt.legend(ncol=2, loc='lower right')\n",
    "\n",
    "plt.gcf().set_size_inches(15, 15)\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "tags": [
     "pdf-inline"
    ]
   },
   "source": [
    "## Inline Question 2:\n",
    "Compare the validation and training accuracies with and without dropout -- what do your results suggest about dropout as a regularizer?\n",
    "\n",
    "## Answer:\n",
    "train_set和val_set直接的gap变小了，这是个有效的正则化手段\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "tags": [
     "pdf-inline"
    ]
   },
   "source": [
    "## Inline Question 3:\n",
    "Suppose we are training a deep fully-connected network for image classification, with dropout after hidden layers (parameterized by keep probability p). If we are concerned about overfitting, how should we modify p (if at all) when we decide to decrease the size of the hidden layers (that is, the number of nodes in each layer)?\n",
    "\n",
    "## Answer:\n",
    "hidden_size变小了，太多drop_out可能会导致训练精度的降低，所以dropout大一点？\n"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.4"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
